IMAGE-BASED SELECTION OF GAMUT MAPPING ^ 

BACKGROUND OF THE INVENTION 
Field Of The Invention 

The present invention relates to gamut mapping of colors in a 
color image, and in particular relates to selection of different gamut 
mappings for different regions of an image based on a spatial frequency 
analysis of the image. 

Description Of The Related Art 

Gamut mapping is a well-known procedure by which colors in 
a color image that are not reproducible by an output device are mapped to 
reproducible colors, while preserving the perception of good color fidelity 
and colorfulness. 

Many different gamut mapping algorithms have been 
proposed, and it is known that some types of color images are gamut- 
mapped better with some gamut mapping algorithms than others. For 
example, it is known that computer graphic images having only a few highly 
saturated colors, such as might be found in a presentation, are gamut 
mapped more accurately by preserving saturation rather than by preserving 
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hue. On the other hand, for natural scene photographic images, better 
gamut-mapped results are obtained by preserving hue and hue gradation. It 
has therefore been considered to analyze an image as a whole, or analyze 
each region in an image, so as to identify images or regions thereof for 
which different gamut mapping algorithms should be applied based on 
5 content. 

It has heretofore been difficult, however, to identify content 
of an image or regions thereof, so as to select an appropriate gamut mapping 
algorithm. In particular, classification of objects in an image, or 
classification of the image itself, often depends on artificial and/or empirical 
10 rules which through trial and error have been found to work on some 

images, but which do not work on all. 



SUMMARY OF THE INVENTION 



15 It is therefore an object of the invention to address the 

foregoing, by applying different gamut mapping algorithms to different 
regions of an original image based on spatial frequency content of the 
original image. 

Thus, in one aspect, the invention is gamut mapping of an 

20 original image by one or more of plural different gamut mapping algorithms, 

in which spatial frequency content of the original image is analyzed, regions 
of the original image are associated with one of the plural gamut mapping 
algorithms based on the spatial frequency analysis, and each region 
associated with a gamut mapping algorithm is gamut mapped using the 

25 associated gamut mapping algorithm so as to produce a gamut mapped image 

of the original image. In preferred embodiments, regions of the image are 
associated with one of the plural gamut mapping algorithms by segregating 
the image into busy regions and smooth regions based on the spatial 
frequency analysis. Gamut mapping proceeds by a first gamut mapping 

30 algorithm (such as a clipping algorithm) for smooth regions, and by a second 



gamut mapping algorithm (such as a compression algorithm) for busy 
regions. In many embodiments, one or more spatial frequency mask may be 
constructed, which mask regions for gamut mapping by the first algorithm 
and regions for gamut mapping by the second algorithm. 

Spatial frequency analysis may proceed by known or hereafter 
developed analysis tools such as discrete cosine transform, discrete Fourier 
transform, or discrete wavelet transform. Regions of the original image are 
associated with one of the plural gamut mapping algorithms based on 
coefficients from such transforms. In addition, the original image may be 
transformed into a color space more suitable for gamut mapping analysis, 
such as a perceptual color space like CIEL*a*b* space or a viewing 
condition independent color space like Jch color space (as defined in 
CIECAM97s). 

Because different gamut mapping algorithms are associated 
with different regions of the original based on spatial frequency analysis of 
the content of the original image, it is possible to select gamut mapping 
algorithms that are more accurately tuned to specific spatial frequency 
content of the original. Thus, better gamut mapping results, inasmuch as 
spatial frequency analysis is defined more concretely in mathematics, and is 
less dependent on artificial rules of object classification as in the prior art. 
In particular, because regions are defined by spatial frequency analysis, as 
are the masks constructed therefrom, the regions need not follow any 
preconceived notions of objects in an image, or of their boundaries. For 
example, prior art image classifiers might successfully identify the image of 
a person in a natural scene photographic image, and segregate the person's 
image as a separate object. The invention, on the other hand, might identify 
several different regions in the person's image, such as hair (which is 
textured), face (which is smooth), clothing, etc., with different gamut 
mapping algorithms being applied against different regions. 

This brief summary has been provided so that the nature of 
the invention may be understood quickly. A more complete understanding 



of the invention can be obtained by reference to the following detailed 
description of the preferred embodiment thereof in connection with the 
attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a view illustrating the outward appearance of 
computing equipment embodying the present invention. 

Figure 2 is a detailed block diagram of the computing 
equipment shown in Figure 1 . 

Figure 3 is a functional block diagram showing a color 
management module implementing image-based selection of gamut mapping 
according to the invention. 

Figure 4 is a detailed flow diagram for explaining the 

invention. 

Figure 5 is a view for explaining color space transformation. 
Figure 6 is a view for explaining wavelet decomposition. 
Figures 7 and 8 are views for explaining normalization of 
wavelet coefficients and rectangularization of the normalized coefficients. 

Figure 9 is a view for explaining construction of a spatial 

mask. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Figure 1 is a view showing the outward appearance of 
representative computing equipment 10 which includes image-based gamut 
mapping according to the invention. Computing equipment 10 includes host 
processor 11 which comprises a personal computer (hereinafter "PC") 
preferably having a windowing operating system such as Microsoft 
Windows, X windows or Macintosh operating systems. Provided with 
computing equipment 10 are color monitor 12 including display screen 14, 



keyboard 15 for entering text data and user commands, and pointing device 
16. Pointing device 16 preferably comprises a mouse, for pointing, 
selecting and manipulating objects displayed on display screen 14. 

Computing equipment 10 includes a computer-readable 
memory medium such as a fixed disk 17 and/or floppy disk drive 19 and/or 
CDROM drive 20. Such computer readable memory media allow computing 
equipment 10 to access information such as image data, computer executable 
process steps, application programs, and the like, stored on removable and 
non-removable memory media. In addition, network access 21 allows 
computing equipment 10 to acquire information, images and application 
programs from other sources, such as a local area network or the internet, or 
from digital cameras or digital video cameras. 

Scanner 22 scans original images such as original color 
photographs or color documents to provide image information to computing 
equipment 10. Printer 24 is a color output device such as an ink jet printer 
or a color laser beam printer. 

Figure 2 is a detailed block diagram showing the internal 
architecture of PC 1 1. As shown in Figure 2, PC 1 1 includes central 
processing unit ("CPU") 25 that interfaces with computer bus 26. Also 
interfacing with computer bus 26 are fixed disk 17, network interface 27 for 
network access 21, random access memory ("RAM") 29 for use as main 
memory, read only memory ("ROM") 30, floppy disk interface 31, CDROM 
interface 32, display interface 34 to monitor 12, keyboard interface 36 to 
keyboard 15, mouse interface 37 to pointing device 16, scanner interface 39 
to scanner 22, and printer interface 40 to printer 24. 

Main memory 29 interfaces with computer bus 26 so as to 
provide RAM storage to CPU 25 during execution of software programs 
such as the operating system, application programs, and device drivers. 
More specifically, CPU 25 loads computer-executable process steps from 
disk 17 or other memory media into a region of main memory 29, and 
thereafter executes the stored process steps from main memory 29 in order 
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to execute software programs. Data such as color images can be stored in 
main memory 29, where the data can be accessed by CPU 25 during 
execution. 

As also shown in Figure 2, fixed disk 17 contains a window 
operating system 41, application programs 42 such as application programs 
that manipulate, obtain and print color images, device drivers 44 such as 
printer driver 45, and a color management module 46 which includes image- 
based gamut mapping according to the invention. 

Image-based gamut mapping according to the invention is 
preferably implemented according to a color management module such as 
that shown at 46. Color management module 46 may be implemented as a 
stand-alone software application program, but more preferably is 
implemented as part of an output device driver such as printer driver 45. It 
is also possible to implement a color management module which includes 
image-based gamut mapping according to the invention, as a dynamic link 
library ("DLL"), or as a plug-in to other application programs such as image 
manipulation programs like Adobe Photoshop. 

Figure 3 is a block diagram showing a preferred 
implementation of image-based gamut mapping embodied in color 
management module 46. As shown in Figure 3, source image data is 
provided to color management module 46 which processes the source image 
data 50 so as to generate destination image data 5 1 . The source image data 
50 is image data in device dependent coordinates, such as RGB image data 
from scanner 22. Destination image data 51 is image data in destination 
device coordinates, such as CMYK coordinates for printer 24. Internally of 
color management module 46, source image data 50 is transformed with the 
source device transform at forward transform 52, so as to yield image data 
in a viewing condition independent color space, such as CIEXYZ or 
CIEL*a*b* space. The transformed image data is converted with an 
appearance model 54 so as to obtain image data in a perceptual color space 
such as Jch color space, or other profile connection space. While in the 



perceptual color space, the image data is subjected to gamut mapping, and in 
particular is subjected to image-based gamut mapping according to the 
invention at 55. Thereafter, the gamut-mapped image data is converted 
through inverse appearance model 56 so as to yield image data in a viewing 
condition independent color space, and thereafter is transformed with an 
inverse device transform 57 for the destination device, so as to yield image 
data in destination device coordinates. 

Figure 4 is a detailed flow diagram for explaining gamut 
mapping according to the invention. Briefly, the process steps shown in 
Figure 4 operate to analyze spatial frequency content of the original image, 
to associate regions of the original image with one of plural gamut mapping 
algorithms based on the analysis of spatial frequency content, and to gamut- 
map each region associated with a gamut mapping algorithm using the 
associated gamut mapping algorithm so as to produce a gamut-mapped 
image of the original image. In addition, the process steps shown in Figure 
4 illustrate construction of a spatial mask which, based on a threshold test 
for values in the spatial mask, masks regions of the original image for gamut 
mapping by a first gamut mapping algorithm and regions of the original 
image for gamut mapping by a second gamut mapping algorithm. 

In more detail, step S401 converts the original image to a 
perceptually uniform space if such a conversion is needed. In particular, 
although the illustrated embodiment shows image based gamut mapping 55 
in the context of a color management module 46, it is possible to implement 
the invention as stand-alone gamut mapping. In such circumstances, it is 
preferred to transform the input image into a perceptually uniform color 
space. Jch color space is preferred, but other color spaces such as 
CIEL*a*b* space are acceptable. 

Figure 5 is a view for explaining the conversion of step S401. 
As shown in Figure 5, an original image 58, sized in this example at 
512x640 pixels, is transformed from RGB space to XYZ space in step s501. 
Step S502 transforms to L*a*b* space, so as to yield an L*-channel color 
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component at 59a, an a*-channel color component at 59b, and a b*-channel 
color component at 59c. 

After conversion of the image to a perceptually uniform color 
space, spatial frequency content of the original image is analyzed. In the 
context of this embodiment of the invention, step S402 analyzes spatial 
frequency content by performing wavelet decomposition on each component 
of the converted image. That is, in a situation where step S401 converts the 
input image to CIEL*a*b* space, then step S402 performs wavelet 
decomposition on each of the L*, the a* and the b* components of the 
converted image. 

Figure 6 shows wavelet decomposition in more detail. For 
each channel 59a, 59b, 59c, wavelet decomposition is performed so as to 
result in recursive representation of the image created by executing a sub- 
sampling filter on the image data. In the preferred embodiment, six levels 
of wavelet data are obtained, with the data taking on values between zero 
and 2 32 in accordance with image content. The six levels are level zero 
through level five. The decomposition of level five data consists of three of 
four bands that each are N/2 x M/2 in size, for an original image sized 
NxM. As shown in Figure 6, these three bands are L*a*b* labelled HH at 
reference numeral 60, LH at reference numeral 61 and HL at reference 
numeral 62. "H" and "L" refer to vertically high or low and horizontally 
high or low pass for the filter that creates these bands. In addition, a fourth 
LL band, indicated generally at 63, consists of all of level four through level 
zero data. Thus, as seen in Figure 6, level four data consists of three of 
four bands that are N/4 x M/4 in size and are L*a*b* labelled respectively 
HH, LH and HL. The LL band is further divided into level three through 
level zero data. Level three data is sub-divided in the same manner as levels 
above it; level two data and below are not shown in Figure 6 for purposes of 
clarity. 

Reverting to Figure 4, steps S403 and S404 operate to 
construct a spatial mask which masks regions of the original image for 



gamut mapping by different ones of plural different gamut mapping 
algorithms. First, in step S403, coefficients from the wavelet decomposition 
performed in step S402 are rectangularized by normalizing the coefficients 
and then plotting the coefficients in row and column order. This is 
illustrated in more detail in Figures 7 and 8 . 

Thus, as shown in Figure 7 for a representative image, 
wavelet decomposition has resulted in coefficient data shown at 64 for HH 
band 60, at 65 for LH band 61, and at 66 for HL band 62. Each of the 
coefficient plots shown at 64, 65 and 66 shows coefficient value in the 
vertical axis as a function of coefficient number in the horizontal axis. In 
this example, the original image was sized at 640x512 pixels; thus, each of 
the HH band 60, the LH band 61 and the HL band 62 is sized at 256x320, 
resulting in 81,920 coefficients in each band, as shown in the plots at 64, 65 
and 66. In Figure 8, the data in each of these plots is normalized 
independently of the other plots, using a linear mapping that results in the 
minimum coefficient value being mapped to a value of zero, and the 
maximum coefficient value being mapped to a value of 255 . Histogram 
analysis many be used to facilitate normalization. After normalization, the 
coefficient data is rectangularized by plotting the coefficients in a matrix 
array sized to the size of the band in question. Thus, since level five data is 
being considered, the coefficient data is plotted in an array sized 256x320, 
yielding the rectangularized coefficients shown respectively at 67, 68 and 
69. Such rectangularized coefficient data is obtained for each level of data, 
from level zero through level 5, and for each component (that is, for the L* 
component, the a* component and the b* component in the case of image 
conversion to CIEL*a*b* space in step S401). 

Reverting to Figure 4, step S401 combines rectangularized 
coefficient data from the levels so as to construct an image mask. This 
process is shown in more detail in Figure 9 which shows rectangularized 
coefficient data for the L* component in CIEL*a*b* space. Figure 9 shows 
rectangularized components 67, 68 and 69 for level five HH data, HL data 
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and LH data, respectively, as explained above in connection with Figure 8. 
In addition, Figure 9 shows rectangularized coefficient data for level five LL 
data 70, which as explained in connection with Figure 6 consists of all of 
levels four through level zero data, each with progressively smaller image 
size. A spatial mask 71 is constructed from the rectangularized coefficient 
5 data. The spatial mask is the same size (MxN) as the original, which in this 

example means that the spatial mask is 512x640 pixels. To construct the 
spatial mask, one pixel from each of the level five HH, LH and HL data is 
extracted from corresponding pixel positions in each of the HH, LH and HL 
rectangularized coefficient data, and each such pixel is inserted into three of 
^ 10 the four pixels in a small 2x2 grid 72 at a corresponding position in the 

--3 spatial mask. The fourth pixel in the 2x2 grid is obtained starting from level 

£3 zero data in the level five band LL data at 70, working up through each of 

i S level one through level four, so as to obtain a value for the fourth pixel in 

'{^ the 2x2 grid. The process is repeated for each corresponding 2x2 pixel grid 

%i 

= 15 in spatial mask 71. 

?*= 

* j While this embodiment obtains the fourth pixel in the 2x2 

grid by working from level zero data up through level four data for the level 

E3 five band LL coefficients at 70, acceptable results have been obtained with 

I* 

different methods for obtaining the fourth pixel. One such method involves 
20 a simple average of the normalized coefficient data from each of the HH, 

HL and LH bands, while ignoring all data from the LL band 70. Other 
methods that have been found to yield acceptable results involve 
combinations of methods between the two extremes of ignoring all level four 
through level zero data, and using all level four through level zero data. 
25 A final spatial mask is obtained by combining the spatial 

masks obtained in accordance with the procedure described for Figure 9 but 
for each of the L* component, the a* component and the b* component (in 
the case of the CIE L*a*b* space). The combination is conducted by a 
weighted Euclidean metric formula, as follows. Each value V in array 72 of 
30 Figure 9 that is used to build the mask, is actually the result of the square 
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root of the sum of the weighted squares of L*, a* and b* wavelet coefficient 
values (rectangularized and normalized as described before). The formula is: 
V = [(k L*)"2 + (a a*)"2 + (p b*) A 2] " (1/2) 



Where L*, a*, and b* are actually the level/band normalized coefficients 
obtained form L*, a*, and b* input data; the constants X, a, and p are 
between 0 and 1, and their values are adjusted in dependence on image 
class. For an image class that includes natural photographic scenes, for 
example, suitable values are X= 1, <x = 1, and p = 1; the default value for all 
three constants is 1 when an image-class rule is not adopted. 

Acceptable results have also been obtained by employing a 
spatial mask constructed solely in accordance with the L* component only, 
or by a spatial mask constructed from a mathematical pixel-for-pixel average 
of the values obtained for the L* component, the a* component and the b* 
component. 

Reverting to Figure 4, step S406 uses the spatial mask to 
associate regions of the original image with one of plural gamut mapping 
algorithms. In this embodiment, two gamut mapping algorithms are used; a 
clipping gamut mapping algorithms and a compression gamut mapping 
algorithms. The clipping gamut mapping algorithm is more suitable for 
regions of an image of low spatial frequency content, because the clipping 
gamut mapping algorithm preserves hue and saturation as much as possible 
although it loses detail. On the other hand, the compression gamut mapping 
algorithm is more suitable to regions of the image with higher spatial 
frequency content, since it tends to preserve high levels of detail, although it 
results in colors that appear more "washed out". Thus, in step S406, a 
pixel-by-pixel inspection is made of the values of the spatial mask. For 
those pixels of the spatial mask that exceed a predesignated threshold, 
indicating a low spatial frequency content, the corresponding pixel in the 
original image is associated with the clipping gamut mapping algorithm. On 
the other hand, for those pixels in the spatial mask whose value does not 
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exceed the predesignated threshold, indicating a high spatial frequency 
content, corresponding pixels in the original image are associated with the 
compression gamut mapping algorithm. Gamut mapping is then performed 
on each pixel of the original image associated with a gamut mapping 
algorithm, using the associated gamut mapping algorithm, so as to produce a 
gamut mapped image of the original. 

Although step S406 uses the spatial mask to select between 
only two gamut mapping algorithms, more than two gamut mapping 
algorithms are possible For example, a selection can be made from three or 
more gamut mapping algorithms, based on a range of values for the pixels in 
the spatial mask, with the gamut mapping algorithms each being tailored for 
a specific range of spatial frequency content. 

Likewise, although the foregoing embodiment results in the 
association of exactly one gamut mapping algorithm for each and every pixel 
of the original image, step S406 can be modified to operate such that pixels 
in a particular spatial frequency range are not gamut mapped at all. 

The invention has been described with respect to particular 
illustrative embodiments. It is to be understood that the invention is not 
limited to the above-described embodiments and that various changes and 
modifications may be made by those of ordinary skill in the art without 
departing from the spirit and scope of the invention. 



